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We propose a novel reflection color model consisting of body essence and (mixed) neuter, and present an 
effective method for separating dichromatic reflection components using a single image. Body essence is an 
entity invariant to interface reflection, and has two degrees of freedom unlike hue and maximum chromaticity. 
As a result, the proposed method is insensitive to noise and proper for colors around CMY (cyan, magenta, 
and yellow) as well as RGB (red, green, and blue), contrary to the maximum chromaticity-based methods. 
Interface reflection is separated by using a Gaussian function, which removes a critical thresholding problem. 
Furthermore the method does not require any region segmentation. Experimental results show the efficacy of 
the proposed model and method. 
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Either extremely diffuse illumination (e.g. hemispheri¬ 
cal lighting) or Lambertian reflectance makes diffuse ap¬ 
pearance of objects. However in many real scenes we 
meet, neither illumination is sufficiently diffuse, nor ob¬ 
ject surfaces exhibit perfectly Lambertian reflection, as 
asserted in pQ. Therefore lots of scene images include 
specular highlights Emu, and detecting and separating 
the specular interface reflection contained in images is 
an important task for object detection and recognition 
[4], and modeling surface reflectance. Since in general, 
specularity is produced by interface reflection, and body 
reflection is diffuse, we do not consider highly diffuse 
interface reflection in this Letter. 

Polarization with the Brewster’s angle can be used 
for imaging interface reflection-reduced appearance in 
the optical process [5|- Numerous techniques have been 
developed to detect and to separate interface reflection 
in the nonoptical process [6;, and many methods have 
employed the dichromatic reflection model [7], which 
is appropriate for the object surfaces of opaque dielec¬ 
tric materials. Under the assumption of dichromatic 
reflection, various color-based methods have been pro¬ 
posed EE- Baiscy et al. presented the S space-based 
color reflection model and a hue-based segmentation al¬ 
gorithm [8]. Their hue-based segmentation algorithm is 
known erroneous in two adjacent uniform-hue regions 
whose saturations are different, as discussed in [9]. Tan 
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and Ikeuchi proposed a maximum chromaticity-based 
separation algorithm [9]. Since their algorithm aligns 
the maximum chromaticities of two adjacent pixels in 
the maximum chromaticity intensity space, the remain¬ 
der of chromaticity (non-maximum chromaticity) may 
not be well aligned. While colors around RGB (red, 
green, and blue) have relatively small values in chan¬ 
nels of non-maximum chromaticity, colors around CMY 
(cyan, magenta, and yellow) rather have two compa¬ 
rable chromaticity values. For example, while a color, 
(0.9,0.03,0.07), which is close to R, has very small val¬ 
ues in G and B channels, another color, (0.47, 0.05, 0.48), 
which is close to M, has two comparable values in R and 
B channels. Moreover the path from diffuse to specu¬ 
lar is nonlinear in their analysis space. Yang et al. also 
used the maximum chromaticity, and further introduced 
the coefficient of variation to make the Ch-CV space for 
efficient separation of reflection components [10 . Their 
algorithm requires segmentation of specular regions, and 
critically depends on the segmentation result. Therefore 
if the segmentation is not successful, the result will be 
undesirable. Shen and Cai used the modified specular- 
free image, in which an offset is added to their specular- 
free image DU- In their method, the diffuse component 
is computed by determining a single parameter that ad¬ 
justs the specularity level in a whole image. Although 
their method is efficient, it needs region segmentation 
(specular/surrounding regions that consist of sufficient 
number of pixels), and ignores variation of diffuse com¬ 
ponent in the specular/surrounding regions. In addition, 
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they did not rigorously justify why their simplification, 
only using the single adjustment parameter in a whole 
image, makes good results for their input images. 

In this Letter, we define mixed reflectance, body 
neuter, body essence, and (mixed) neuter, and propose 
a novel reflection color model, BREN (body reflection 
essence-neuter) model, where mixed reflectance consists 
of body essence and mixed neuter. Based on BREN 
model, we present an effective method for separating 
dichromatic reflection components using a single image. 
Body essence is an entity invariant to interface reflection, 
and has two degrees of freedom unlike hue and maxi¬ 
mum chromaticity. As a result, the proposed method is 
insensitive to noise and proper for colors around CMY 
as well as RGB contrary to the maximum chromaticity- 
based methods. The method computes local gradients 
of mixed neuter and body essence, and interface reflec¬ 
tion is separated by using a Gaussian function, which 
removes a critical thresholding problem. Moreover the 
method uses neither any specular-free image nor any 
modified one, and does not require any region segmen¬ 
tation. 

Now we present the body reflection essence-neuter 
model for separating reflection components. In the 
dichromatic reflection model, a reflected irradiance L 
consists of interface (f) and body (b) reflection compo¬ 
nents, given as: 

L = L f + L b = rafCf + ra b G b , (1) 

where m f and ra b are the geometric terms, and Gf = 
(C R , C G , C B ) and G b = {C R ,C G ,C B ) are the spectral 
terms. From Eq. [l] each channel’s irradiance can be 
given by 

L 1 = rafC/ + ra b G b , ( 2 ) 


where /, J = R,G,B. For highlight removal, we need 
a known entity invariant to interface reflection. From 
Eqs. [ 6 ] and [7| it is known that body essence is invari¬ 
ant to interface reflection (has no portion of interface 
reflection), and of two degrees of freedom (it is of three 
channels and zero mean) unlike hue and maximum chro¬ 
maticity (both, one degree of freedom). 

From Eqs. [5] and [7| 

pi = S N + TJ + S 1 . (8) 

Hence we get 

P 1 -S 1 = S N + t 7 . (9) 

Since S N + rj is constant with respect to I = R,G,B, 
P 1 — S 1 is spectrally neutral. Thus we define that entity 
as (mixed) neuter, 

V = P T -<S 7 , (10) 

and we get Lemma [l] 

Lemma 1 (Mixed neuter). Let P 1 and S 1 where I 
is a spectral channel index (e.g. I = R,G,B) be the 
mixed reflectance and body essence, respectively, under 
dichromatic reflection assumption. Then V = P 1 — S 1 
is constant with respect to I, i.e. spectrally neutral. 

Proof. From the definitions of the mixed reflectance and 
body essence, P 1 = S N + S 1 , and S 1 = S 1 — 

where S N = mfS N , S 1 = m b S' / , n c is the number of 
color channels, and J is a spectral channel index. Hence 
P 1 — S 1 = S N + — V j S J , thus P 1 — S 1 is constant 
with respect to /, i.e. spectrally neutral. □ 

From Eq. [lOj we meet the core of BREN model, a 
novel intuitive expression of dichromatic reflection, 


where / = R,G,B. The spectral term can be ex¬ 
pressed by illumination E = (E R ,E G ,E B ) and inter¬ 
face/body spectral reflectances Sf = (S N , S N , S N ) and 
Sb = (S R ,S G ,S B ), and hence 

L 1 = mfS N E 1 + mbS^ 1 . (3) 

We define the mixed reflectance as 

P 1 = = mfS N + nibS 1 . ( 4 ) 

Letting S N = mfS N (interface reflectance) and S 1 = 
mbS 1 (body reflectance) gives 

P 1 = S N PS 1 . ( 5 ) 

Now we define body neuter (non-negative) as 

v = ( 6 ) 

j 

and body essence as 

S^S 1 - rj , (7) 


P 1 = V + S 1 , ( 11 ) 

which demonstrates that the mixed reflectance is the 
sum of the body essence and (mixed) neuter. 

Mixed neuter can be easily computed from a mixed re¬ 
flectance. Lemma [2] presents how to compute the mixed 
neuter given a mixed reflectance. 

Lemma 2 (Computation of mixed neuter). Let 

P 1 and S 1 where I is a spectral channel index (e.g. 
I — R,G,B) be the mixed reflectance and body essence, 
respectively, under dichromatic reflection assumption. 
Then V = P 1 — S 1 is the mean of P 1 with respect to 
I, i.e. 

p = ^E p '. < 12 ) 

where n c is the number of color channels. 

Proof. Summation of Eq. [10] with respect to I gives 
n c V = ^2jP 1 — 22 i^ T ' Since body essence is of 
zero mean as presented earlier (Eqs. [ 6 ] and [T]) , V — 

p~ c T,i p 1 - □ 
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Actually Lemma 2 implies Lemma 1 . Since the mixed 
neuter is equivalent to the mean of mixed reflectance 
with respect to /, it is constant with respect to /. Ac¬ 
cording to Eqs. [lT]and[T2j body essence can be directly 
calculated from the mixed reflectance: 

S 1 = P 1 - V, (13) 

and so it can be considered a known entity if the mixed 
reflectance is known (this is one of usual assumptions in 
separation of reflection components from a single image). 

We assume that the illumination is known or properly 
estimated. One trivial scheme to estimate illumination 
from a single image is averaging each channel’s inten¬ 
sity for all pixels. With Eq. [4j the mixed reflectance is 
calculated for each pixel from the RGB intensities and 
illumination. Then based on the mixed reflectance, the 
mixed neuter and body essence are calculated for each 
pixel by Eqs. [ 12 ] and [l3j 

In most cases, only a small portion of an image has 
specular highlights. Therefore we mostly do not need 
to consider all pixels in an image for highlight removal. 
Consequently we consider only high-neuter pixels, prac¬ 
tically, pixels whose mixed neuter is larger than a thresh¬ 
old. If body reflectance ( S 7 ) is constant in a finite region, 
in the region, A S N = A P 1 from Eq. [HJ 77 and S 1 are con¬ 
stant from Eqs. [6] and [7| hence A S N = A P 1 = AV from 
Eq. [TT] That is, reducing the mixed neuter is equiva¬ 
lent to reducing the interface reflection component for 
a region of constant body reflectance. For that rea¬ 
son, we use the mixed neuter to reduce the interface 
reflectance since interface reflectance is unknown. For 
each high-neuter pixel, we consider iterative highlight 
removal, given as 

'Pk+i = 'Pk + (14) 

where k denotes the iteration number. The above equa¬ 
tion iteratively reduces the mixed neuter (say, neuter 
demotion) whenever AVk < 0 . AVk is determined as 
follows. For the high-neuter pixel, we compute the gra¬ 
dients of mixed neuter along the eight-connected pixels: 

A i V = V(p i )-V(p), (15) 

where p is the high-neuter pixel location, and is 
an eight-connected pixel location of p. One may 
choose other types of neighborhood relations (e.g. four- 
connectivity) instead of eight-connectivity, tradeoffing 
the computational cost and possibility of finding good 
neighbors. 

We assume that if S 1 is constant in a region, S 1 is 
also constant in the region (hence so is 77). Therefore 
we use S 1 to evaluate the closeness of any two body re¬ 
flectances since S 1 is unknown. Table [l] shows constan¬ 
cies and changes of related entities in changes of shading 
and specularity (interface reflection). From the table, it 
is known that in specularity change, unlike in shading 
change, body essence is completely suitable for identify¬ 
ing body reflection colors rather than chromaticity (or a 


Table 1. Change and constancy of reflection from a surface 
with constant reflectance 



Shading change 

Specularity change 

Constancy 

Chromaticity 

Body essence 

Change 

Luminance 

Mixed neuter 



(a) (b) (c) (d) 


Fig. 1. Results of Head and Dinosaur, (a) Input Head 
image (R. T. Tan’s) and (b) the result, (c) Input Dinosaur 
image and (d) the result. 

portion of it such as hue and maximum chromaticity) is. 
Three entities, hue, maximum chromaticity, and body 
essence are all invariant to interface reflection. However 
body essence has two degrees of freedom by its definition 
while hue and maximum chromaticity have only one de¬ 
gree of freedom. Therefore, body essence provides more 
information of body reflection than the other two enti¬ 
ties do. Since body essence is of two degrees of freedom, 
it is more insensitive to noise than hue and maximum 
chromaticity are. Furthermore, body essence is proper 
for colors around CMY as well as RGB, contrary to 
maximum chromaticity, which by its definition cannot 
contain multiple comparable values from color channels 
simultaneously. 

Among the eight-connected pixels, we only consider 
ones whose mixed neuter is smaller than the mixed 
neuter of the current pixel, and compute the gradients 
of body essence along the eight-connected pixels: 

Aj<S = <S(pj) — <S(p), (16) 

where S = (S R , S G , S ,3 j. Then we set A Vk for the mini- 
mum of gradient of mixed neuter weighted by a Gaussian 
of the essence similarity: 

AVk = min e -A||A;<s fc || AiVk • (17) 

ie{i\AiV k <o} 

The above Gaussian function replaces a conventional 
thresholding operation, and provides more appropriate 
use of body reflectance similarity in highlight removal. 

We have tested the proposed method for various 
images, and provide results of five input images. 
Three input images (Head, Fish, and Toys) are down¬ 
loaded from a webpage of R. T. Tan (http://php- 
robbytan.rhcloud.com/code.html), and the other two 
images (Dinosaur and Mickey-ball) are newly captured. 

Figure [T^i-b show the result of a single-colored object, 
the Head. We can see that the specular component is 
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(a) (b) 

Fig. 2. Result of Mickey-ball, (a) Input image and (b) 
result. 


(a) (b) 

Fig. 3. Result of Fish, (a) Input image (R. T. Tan’s) and 
(b) result. 

adequately removed. Figure [lj^-d show the result of the 
Dinosaur. Even though there is smooth color gradation 
in its tail, the specular component is removed quite well 
without undesirable artifact. 

Figures ii and [4] show the results of more compli¬ 
cated scenes, the Mickey-ball, Fish, and Toys, respec¬ 
tively. Despite their high complexity of colors and tex¬ 
tures, the body reflection components are properly esti¬ 
mated. 

We presented a novel reflection color model (BREN) 
and an effective method for separating dichromatic re- 


(a) (b) 

Fig. 4. Result of Toys, (a) Input image (R. T. Tan’s) and 
(b) result. 


flection components using a single image. We showed 
body essence is a better entity for specular highlight re¬ 
moval than hue and maximum chromaticity are. The 
Gaussian coefficient generalizes a conventional simple 
thresholding scheme, and it provides detailed use of 
body color similarity. The proposed method does not 
require any region segmentation, and thus it does not 
depend on segmentation accuracy. 

The authors are grateful to Professor Robby T. Tan 
for his images we have used here. This research was sup¬ 
ported by Basic Science Research Program through the 
National Research Foundation of Korea (NRF) funded 
by the Ministry of Education (No. 2012R1A1A2009138). 
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